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ABSTRACT 


Many  problems  in  operations  research  require  the  maximization  or  mini¬ 
mization  of  a  suitable  payoff  function  subject  to  various  constraints.  Lagrange 
multipliers  are  classically  used  for  this  type  of  problem,  however,  the  treat¬ 
ment  given  this  technique  by  most  texts  requires  that  the  payoff  and  constraint 
functions  be  at  least  differentiable  at  the  extremizing  point.  This  paper  shows 
that  the  Lagrange  multiplier  concept  can  be  independent  of  differentiability  or 
even  continuity  of  the  functions  involved.  It  also  gives  the  reader  a  geometric 
insight  into  the  workings  of  the  multiplier.  Simplifications  possible,  if  the 
functions  involved  are  homogeneous, are  displayed.  The  treatment  is  heuristic 
rather  than  formally  rigorous. 
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I.  INTRODUCTION 


Many  problems  in  operations  research  require  the  maximization  or  mini¬ 
mization  of  a  suitable  payoff  function  subject  to  various  constraints.  I  ^grange 
multipliers  (LMs)  are  classically  used  for  this  type  of  problem,  however, 
the  treatment  given  this  technique  by  most  texts  requires  that  the  payoff  and 
constraint  functions  be  at  least  differentiable  at  the  extremizing  point.  This 
paper  indicates  that  the  LM  concept  can  be  independent  of  differentiability 
or  even  continuity  of  the  functions  involved.  The  ideas  presented  follow 
closely  those  of  Hugh  Everett  in  reference  (a).  The  reader  is  given  a  geometric 
insight  into  the  workings  of  the  LM  .  Simplifications  which  may  result, 
if  the  functions  involved  are  homogeneous,  are  displayed.  The  general  treat¬ 
ment  is  heuristic  rather  than  formally  rigorous.  Many  examples  (see  appendix) 
have  been  worker’  out. 
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II.  THE  DIRECT  METHOD  -  MAIN  THEOREM 

The  LM  concept  retains  utility  even  when  the  functions  involved  are  al¬ 
most  completely  unrestricted  in  character.  The  technique  for  using  the  LM 
in  this  context  will  be  referred  to  as  the  ’’direct  method"  in  contrast  to  the 
"classical  method"  which  involves  setting  partial  derivatives  equal  to  zero. 

It  is  desired  to  maximize  (or  minimize)  a  function  H(x)  subject  to 
constraints  g.  (x)  =  C.,  i  =  1, .  .  .,m.  Except  for  being  real  valued,  no 
restrictions  are  imposed  on  these  functions.  The  variable  "x"  may  have  any 
set  whatever  as  domain;  this  set  will  be  called  the  "strategy  set"  and  will 
be  designated  "S".  The  theorem  which  follows  sets  the  stage  for  the  use  of 
the  LM  in  solving  this  type  of  problem . 

Theorem:  (direct  method)  Let  L,  i  =  l . m  be  fixed,  arbitrarily 

chosen,  real  numbers.  Define  F(x)  as  follows: 

F(x)  =  H  (x)  -  Y  Xjgj  (x) 

Suppose  that  x'  is  a  point  in  S  which  maximizes  F(x)  in  comparison  with 
all  other  points  in  S  .  Then  x'  maximizes  H(x)  in  comparison  with  all 
the  points  x  in  S  which  fulfill  the  constraints  g^(x)  =  g.(x')  , 

Proof:  By  definition  of  x*  ,  F(x')  £  F(x)  for  all  x  in  S  .  Substituting 
for  F(x): 

H(x’)  -£\.g.(x’)  s;  H(x)  -  ^X.g.(x)  for  all  x  in  S  . 

If  attention  is  restricted  to  all  x  in  S  which  fullfill  the  constraints 
gA(x)  =  gi(x’)  for  all  values  of  i,  then  the  summations  are  equal  and  can  be 
cancelled  from  the  inequality,  leaving  H(x’)  £  H(x)  for  these  particular 
x  in  S,  which  was  to  be  proved. 

The  following  comments  are  designed  to  highlight  the  important  points  in 
the  theorem,  and  to  anticipate  questions  which  may  have  occurred  to  the 
reader. 

Comment  1:  The  numbers  X.f  i  =  1, .  .  ,,m  are  the  LMs  and  are  fixed 
numbers  which  are  chosen  before  the  maximization  is  accomplished.  This 
is  in  sharp  contrast  to  the  classical  method  in  which  the  LMs  are  left  in  the 
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form  of  undetermined  parameters  until  the  end  of  the  process.  This  is  one  of 
the  prices  that  the  direct  method  must  pay  because  it  allows  the  functions 
involved  to  be  non- differentiable.  In  special  cases  it  is  possible  to  retain  the 
LMs  as  parameters  in  the  direct  method  also  (see  section  VIII). 

^H!?.g.nt J2;  F(x)  is  written  in  the  form  H(x)  -  ^A.g.(x).  This  is  also  the 
way  in  which  LMs  are  used  in  the  classical  method.  Other  forms  of  F(x) 
are  possible  in  both  methods  (section  VIII). 

Comment  3:  The  theorem  states  "Suppose  that  x'  is  a  point  in  S  which 
maximizes  F(x).  .  .".  Note  that  F(x)  is  to  be  maximized,  specifically. 

The  finding  of  an  inflection  point  or  a  relative  maximum  will  not  do.  However, 
there  is  no  guarantee  that  a  maximizing  point,  x’  ,  exists  in  S  .  The  theorem 
is  applicable  only  when  such  a  point  exists . 

Comment  4;  By  merely  reversing  the  inequalities,  the  proof  of  the  correspond¬ 
ing  theorem  for  the  minimum  of  H(x)  is  obtained. 

Comment  5:  The  theorem  proves  that  x'  yields  the  absolute  maximum  of 
H(x) ,  as  opposed  to  a  relative  maximum . 

Comment  6:  The  point  x'  is  _a_  point  in  S  which  maximizes  F(x) .  In  case 
there  are  several  such  points,  the  proof  still  holds. 

Comment  7;  The  point  x'  is  obtained  by  maximizing  F(x)  without  regard  to 
the  values  taken  by  the  constraint  functions.  This  is  precisely  why  the  LMs 
are  valuable;  they  turn  a  maximization  problem  with  constraints  into  a 
maximization  problem  without  constraints. 

Comment  8:  The  theorem  says  that  x'  maximizes  H(x)  subject  to  the 
constraints  gj(x)  =  g^x') .  The  original  problem  was  to  maximize  H(x) 
subject  to  g.(x)  =  C.  .  There  is  no  assurance  that  g.(x')  =  C.  for  any  or  all 
values  of  i,  hence  the  original  problem  has  not  been  solved  yet.  Recall  that 
the  LMs  were  chosen  arbitrarily.  It  may  be  possible  to  cause  g.(x')  =  C. 

for  all  values  of  i  by  suitably  modifying  the  LMs  and  repeating  the  process. 

If  this  is  possible,  the  direct  method  solves  the  original  problem .  In  many 
OR  problems,  however,  one  is  not  interested  in  maximizing  H(x)  subject 
to  specific  constraints,  but  rather  in  exploring  the  entire  range  of  what  can 
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be  obtained  as  a  function  of  the  constraint  values,  C.  .  In  this  case  the 

1 

process  of  sweeping  through  a  range  of  values  of  the  LMs  may  be  completely 
adequate. 

The  direct  method,  then,  boils  down  to  the  following  sequence: 

Step  1:  Pick  the  LMs  arbitrarily.  Section  III  will  provide  guidance  for 
correcting  this  initial  guess,  if  necessary. 

Step  2:  Form  F(x)  and  maximize  it  (or  minimize,  if  desired),  obtaining  a 
maximizing  point  x' ,  and  corresponding  values  g.(x')  of  the  constraint 
function. 

Step  3:  If  g^x')  =  C.  for  all  values  of  i  =  1, .  .  ,,m  then  the  original 
problem  is  solved  and  the  maximum  value  of  H(x),  subject  to  g.(x)  =  C., 
is  given  by  H(x')  .  If  g.(x')7*  C.  for  all  values  of  i,  it  is  necessary  to 
pick  new  values  of  the  LMs  and  repeat  the  process,  aiming  for  convergence 
of  the  values  g.(x')  to  C.  . 

An  example  of  the  direct  method  is  given  in  the  appendix  (example  1). 

H(x)  and  g(x)  are  differentiable,  but  S  is  a  collection  of  discrete  points, 
and  the  classical  method  does  not  work. 
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III.  INTERPRETATION  OF  THE  LAGRANGE  MULTIPLIER 
A.  The  g, H  plane 

This  section  is  intended  to  promote  geometric  insight  as  towhy  the  LM  works. 
One  constraint  function,  g(x),  will  be  used  in  this  discussion,  but  the  results  hold  for 
any  number  of  constraints.  Letting  x'  denote  a  point  in  the  set,  S,  which 
maximizes  H(x)  -  Xg(x)  in  comparison  with  all  other  points  in  S,  we  have 
F(x')  =  H<x')  ~  *g(x')  *  H(x)  -  Xg(x)  for  all  x  in  S  .  Solving  thii  for  H(x): 
H(x)  *  g(x)  +  F(x’)  for  all  x  in  S 

From  this  basic  inequality,  most  of  the  results  of  this  section  will  follow 
immediately.  It  will  be  useful  to  look  at  what  is  happening  in  the  g,H  plane. 


H 


-5- 


As  depicted  in  figure  1,  to  each  point  in  S  there  corresponds  a  point 
(g(x),H(x))  in  the  g,H  plane.  In  this  manner  the  entire  strategy  set,  S,  is 
mapped  into  some  set,  called  the  payoff  set,  in  the  g,  H  plane.  This  payoff 
set  is  shown  as  the  shaded  region  in  figure  1.  In  general  there  is  no  way  of 
predicting  just  what  this  payoff  set  is  going  to  look  like. 

The  inequality  H(x)  ^  Xg(x)  +  F(x')  for  all  x  in  S  says  that  all  the 
points  in  the  payoff  set  lie  below  the  line  whose  equation  is  H  =  Xg  +  F(x’) . 
Futhermore,  it  says  that  this  line  touches  the  payoff  set  only  at  the  maximizing 
point  (or  points,  in  case  of  non-unicity).  This  immediately  suggests  that  the 
act  of  guessing  a  value  for  X  in  step  1  of  the  direct  method  is  no  more  than 
guessing  the  slope  of  a  line  in  the  g,  H  plane.  The  act  of  maximizing  F(x), 
step  2,  to  find  F(x),  simply  pushes  the  H-intercept  up  the  H  axis  until  the 
line  H  =  Xg  +  F(x')  just  touches  the  payoff  set  at  the  point  (g(x’)t  H(x’))  as 
shown  in  figure  2 .  In  brief,  X  is  the  slope  of  the  line,  and  the  F(x’)  is  the 
H-intercept. 


FIGURE  2 
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Figure  2  also  illustrates  how  guessing  new  values  of  A.  produces  new 
values  of  the  constraint  function  g(x)  .  If  the  value  g(x’),  initially  obtained 
by  steps  1,2,  is  not  large  enough,  then  A  must  be  made  smaller,  i.e.  the 
slope  of  the  line  must  be  decreased.  The  resulting  line  is  illustrated  in  the 
figure  above  as  H  =  A'g  +  F(x")  .  It  is  obvious  that  the  corresponding  value 
g(x")  cannot  possibly  be  less  than  g(x’).  This  basic  concept  will  help  the 
analyst  modify  his  guesses  of  A  in  the  proper  direction  to  obtain  the  desired 
constraint  values . 

Figure  2  illustrates  another,  very  important  point.  Suppose  that  after 
choosing  a  value  for  the  LM  we  obtain  a  maximizing  point  x'  and  a 
corresponding  value  g(x')  of  the  constraint.  We  know  from  the  main  theorem 
that  this  x'  maximizes  H(x)  subject  to  the  constraint  g(x)  =  g(x’)  .  Suppose 
further  that  the  value  of  the  LM  chosen  was  positive.  Then  figure  2  illustrates 
that  you  cannot  do  better,  i.e.,  obtain  a  larger  H(x),  by  using  any  smaller 
value  of  constraint  than  g(x')  .  This  follows  directly  from  the  basic  inequality 
obtained  at  the  beginning  of  this  section,  and  is  one  of  the  main  advantages  of 
using  the  linear  form  of  F(x)  .  Again,  if  the  LM  is  positive,  then  x' 
maximizes  H(x)  subject  not  only  to  g(x)  =  g(x'),  but  also  to  g(x)  <;  g(x’)  . 

This  is  of  paramount  importance  in  problems  where  g(x)  is  a  money  or  other 
resource  constraint,  and  H(x)  is  the  payoff  for  expending  g(x)  units  of  the 
resource. 

B.  Definition  of  maxH,  minH 

After  the  maximization  operation  is  completed  successfully,  i.e.,  a  point 
x'  is  found  which  maximizes  H(x)  subject  to  g(x)  =  C,  it  is  intuitively  clear 
that  x  depends  upon  the  value  of  C  .  (Otherwise,  the  constraint  is  having  no 
effect  on  the  problem  and  may  as  well  be  dispensed  with.)  It  follows  that 
H(x  ) ,  the  resulting  maximum  value  of  H(x),  is  also  a  function  only  of  C  , 

This  function  is  denoted  maxH(C),  and  the  corresponding  function  resulting 
from  minimizing  will  be  denoted  minH(C).  Example  2  displays  a  typical 
H(x)  and  the  resulting  minH(C). 
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These  functions  have  an  important  interpretation  in  the  g,H  plane.  The 
line  in  the  g,  H  plane  whose  equation  is  g  =  C  is  assumed  to  intersect  the 
payoff  set  as  shown  below. 


FIGURE  3 

The  points  in  the  payoff  set  which  fall  on  this  line  are  precisely  those  for 
which  g(x)  =  C  .  The  point  where  the  line  g  =  C  cuts  the  upper  boundary  of 
the  payoff  set  is  the  desired  maximum  value  of  H(x)  subject  to  the  constraint 
g(x)  =  C  .  It  follows,  therefore,  that  the  entire  upper  boundary  of  the  payoff 
set  is  the  graph  of  the  function  maxH(C)  (the  lower  boundary  is  the  graph  of 
minH(C))  provided  only  that  the  axes  in  figure  3  are  relabled  maxH(C),C  instead 
of  H, g  respectively.  It  is  emphasized  that  H(x)  and  maxH(C)  are  completely 
different  functions .  The  properties  of  these  two  functions  may  be  strikingly 
different,  as  will  be  illustrated  in  the  following  subsection. 


Figure  4  displays  the  typical  situation  resulting  from  the  successful  use 
of  the  direct  method.  For  simplicity  of  notation  and  graphing  only  one 
constraint  is  used. 
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H(x' 


F(x‘ 


FIGURE  4 

A  maximizing  point  x'  has  been  found  which  maximizes  F(x).  Also 
g<x’)  =  C.  the  desired  value.  Assume  that  the  function  maxH  is  differentiable 
at  this  value  C.  If  so,  the  line  H  =  kg  +  F(x')  and  the  function  maxH  must 
have  the  same  slope  at  the  point  where  the  line  touches  the  payoff  set.  The 
slope  of  the  line  is  \  and  the  slope  of  maxH  at  this  point  is  d(maxH)/dc 
(partial  derivative  in  the  case  of  multiple  constraints).  This  is  the  desired 

interpretation  of  the  LM .  The  LM  is  the  derivative  of  maxH(C)  with 
respect  to  C . 

This  interpretation  of  the  LM  will  frequently  permit  a  ballpark  guess 
as  to  the  proper  value  of  the  LM  when  using  the  direct  method.  This  approach 

is  particularly  useful  when  the  functions  maxH(C),  H(x),  and  g.(x)  have 
physical  or  geometric  interpretations  (example  3). 

Algebraic  constraints  may  usually  be  formulated  in  a  number  of 
equivalent  ways .  For  example,  g(x, y)  =  ax2 +bxy +cy2  =  D  can  be 
rewritten  as  (ax  +  cy2  -  D)/xy  =B  .  In  the  first  case  the  LM  will  be 
d(maxH)/dD,  in  the  second,  d(maxH)/dB  .  Since  the  effect  of  varying  D 
is  to  expand  or  contract  the  geometric  figure  g(x,y)  =  D  while  varying  B 
rotates  the  geometric  figure,  the  numerical  value  of  the  LM  will  be  different 
in  the  two  cases.  As  a  result,  an  apparently  trivial  reformulation  of  a 
constraint  may  introduce  or  remove  a  pathology  from  the  problem. 
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The  foregoing  interpretation  of  the  LM  as  a  partial  derivative  depends 
upon  maxH(C)  being  differentiable.  Frequently  this  is  not  the  case;  this  can 
occur  in  various  ways.  One  of  the  simplest  is  for  the  strategy  set  S  to  have 
only  a  finite  number  of  points,  as  in  example  1.  H(x)  is  differentiable  through¬ 
out  most  of  the  region  of  interest,  but  the  payoff  set  contains  only  five  points, 
hence  maxH(C)  is  nowhere  differentiable.  On  the  other  hand,  it  is  possible 
for  maxH(C)  to  be  differentiable  even  when  some  or  all  of  the  functions  H(x) 
and  gA(x)  are  non -differentiable.  In  example  8,  g(x)  is  discontinuous  at  the 
minimizing  point  for  all  values  of  C  greater  than  -  1.90,  yet  d(minH)/dC 
exists  and  equals  X  for  all  such  values  of  C  . 
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IV.  VARIOUS  PATHOLOGIES 


Frequently  there  are  several  values  of  x  in  S  which  will  maximize  F(x) 

for  a  particular  value  of  the  LM.  One  common  cause  of  this  is  illustrated 
below. 


FIGURE  5 

The  payoff  has  a  concavity  in  the  upper  envelope,  allowing  the  line 
H  =  Xg+  F(x’)  to  be  tangent  at  two  places.  This  illustrates  another  point. 

It  is  impossible  to  find  a  LM  which  will  give  rise  to  values  between  and 
C2  without  violating  the  inequality  H(x)  s  Xg(x)  +  F(x’),  which  was  obtained 
at  the  beginning  of  section  III.  Such  regions  are  called  "gaps”,  characterized 
by  a  small  change  in  the  LM  causing  a  large  jump  in  the  resulting  value  of  the 
constraint.  One  way  of  overcoming  the  effect  of  a  gap  will  be  discussed  in 
section  VIII;  another  method  is  mentioned  in  reference  (a). 

In  figure  5,  above,  two  values  of  x  in  S  are  obtained  which  maximize 
F(x);  one  gives  the  value  Cy  the  other,  C2>  It  is  possible  for  there  to  be  an 
infinite  number  of  points  in  S  which  maximize  F(x)  for  a  certain  value  of  the 
LM.  The  most  common  cause  for  this  phenomenon  is  illustrated  in  figure  6, 
next  page . 
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Between  Cj  and  C2  the  function  maxH(C)  is  a  straight  line  segment 
with  slope  X.  All  of  the  points  x'  in  S  which  map  onto  this  line  segment 
will  maximize  F(x)  for  this  particular  value  of  the  LM,  producing  the  same 
F(x').  This  phenomenon,  the  "constant  multiplier",  will  probably  be  fatal  if 
a  computer  is  being  used  to  solve  the  problem,  unless  the  program  is  care¬ 
fully  written.  Unless  exactly  the  correct  value  of  the  LM  is  guessed,  it  will 
not  be  possible  for  the  computer  to  arrive  at  any  value  of  C  between  Cj  and 
C2-  On  the  other  hand,  if  the  exact  value  of  the  LM  is  guessed,  the  computer 
may  try  to  print  out  all  the  points  in  S  which  maximize  F(x)  for  this  value 
of  the  LM.  Hence  some  care  must  be  taken  in  the  programming  of  the  direct 
method  to  test  for  non-unicity  of  x*  before  ordering  a  printout  (example  4). 

The  preceeding  figures  illustrate  that  one  value  of  the  LM  may  give  rise 
to  many  maximizing  points  x’.  The  opposite  situation  may  occur,  many 
values  of  the  LM  may  give  rise  to  the  same  x'.  This  situation  is  very  com¬ 
mon,  arising  primarily  by  maxH(C)  not  being  differentiable.  Figure  7,  next 
page,  illustrates  how  this  may  occur. 
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H 


FIGURE  7 


The  function  maxH  has  a  sharp  corner  at  one  point,  i.e.  maxH  is  not 
differentiable  there.  It  is  clear  that  there  are  many  slopes  such  that  the 
resulting  line  will  touch  the  payoff  set,  as  shown  in  figure  7.  All  these  values 
of  the  LM  will  give  the  same  value  for  g(x’)  and  for  H(x'). 
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V.  THE  CELL  PROBLEM 


There  is  a  type  of  problem,  the  cell  problem,  for  which  the  LM  technique, 
direct  or  classical,  is  particularly  well  suited.  It  is  characteristic  of  this 
problem  that  F(x)  can  be  written  in  the  form 

DW  -  •  &<V 


In  other  words,  in  the  cell  problem  we  have  H(x)  -  T^H-fx.)  and  g(x)  =^g.(x.) . 
For  simplicity  of  notation  only  one  such  constraint  is  shown  in  F(x),  but  any 
number,  each  with  its  own  LM,  is  permitted,  as  usual.  It  is  clear  that  F(x) 
can  be  rewritten  as  a  single  sum 

DW  -  WV) 


of  "n"  terms,  hence  the  name .  It  follows  that  F(x)  can  be  maximized  by 
maximizing  each  cell  separately  (using  the  same  value  of  LM),  and  summing 
the  results.  A  computer  is  especially  useful  for  cell  problems  involving  a 
large  number  of  cells  because  computing  time  increases  linearly  with  the 
number  of  cells  rather  than  exponentially.  Even  if  the  maximization  in  each 
cell  is  done  analytically  the  computer  is  useful  in  summing  results  for  each 
value  of  LM,  and  in  otherwise  "keeping  the  books". 

Typical  problems  of  this  type  are  weapon  allocation  or  weapon  mix 
problems.  Example  4  is  of  this  type.  Reference  (a)  discusses  cell  problems 
in  some  detail.  Reference  (b)  contains  an  example  of  a  cell  problem  in  which 
the  casuality  rates  in  several  shipping  channels  due  to  mine  laying  is  maximized 
subject  to  constraints  on  the  number  and  types  of  mines  laid. 
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VI.  COMPARISON  OF  THE  DIRECT  AND  CLASSICAL  METHODS 


It  is  desired  to  maximize  via  the  classical  method  a  function  H(x)  subject 
to  constraints  g.(x)  =  C.,  j  =  1,2 . m.  Here,  "x"  is  a  point  in  Euclidean 

J  J 

"n"  space,  i.e.,  "x"  denotes  (x^ . x^).  The  notation  "C"  and  "X"  will 

similarly  be  used  to  denote  the  "m"  space  vectors  (C C  )  and  (X . .  X  ). 

1  m  1  m 

This  is  in  contrast  to  the  direct  method  where  there  are  no  restrictions  on  the 
character  of  the  argument  of  H(x).  As  normally  presented,  the  classical 
method  consists  of  the  following  steps: 

Step  1:  Form  F(x)  =  H(x)  -^X.g.(x)  where  the  LMs  ai  parameters  rather 
than  specific  values. 

Step  2:  Write  down  the  system  of  "n"  equations 

Fx.  ■  0 
1 

where  F  denotes  the  partial  derivative  of  F  with  respect  to  x., 

l  1 

evaluated  at  "x" . 

Step  3:  These  equations  plus  the  constraint  equations  provide  m+n  equations 

in  the  m+n  unknowns,  x. ,  . . . ,  x  ,  X, ,  . . . ,  X  .  Solve  these  equations  for  the 

l  n  l  m 

unknowns.  The  resulting  solutions  (there  may  be  several)  represent  stationary 
points  for  H(x)  which  must  be  tested  separately  to  determine  which  represent 
relative  extrema,  stationary  points  only,  or  absolute  extrema.  Example  5 
illustrates  this  method. 

The  following  comments  are  intended  to  clarify  classical  methods  and  to 
help  pinpoint  the  differences  between  the  two  methods: 

Comment  1:  In  step  1,  the  LMs  are  parameters  rather  than  specific  numbers. 

In  order  for  this  to  work,  the  maximizing  point  x*  must  be  a  smooth,  continuous 
function  of  both  C  and  X.  This  is  because  step  3  is  equivalent  to  solving  for 
the  maximizing  vector  x'  as  this  function  of  C  and  X,  then  using  the  constraint 
equations  to  eliminate  X.  When  non-differentiable  functions  are  used,  the 
function  x'(C,  X)  may  not  exist,  or  be  expressible  in  any  reasonable  form. 
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Comment  2:  Step  2  is  analagous  to  maximizing  or  minimizing  a  function  of  one 
variable  by  setting  its  derivative  equal  to  zero.  In  the  event  that  this  procedure 
locates  the  absolute  maximum  or  minimum  of  F(x)  then  the  main  theorem  of 
section  I  provides  a  proof  for  the  method . 


Comment  3:  Since  the  absolute  maximum  of  F(x)  is  not  the  goal  of  steps  2,3, 
the  inequality  of  section  III  does  not  hold  in  the  classical  method.  This  has  two 
immediate  consequences: 

(a)  The  pathologies  of  section  IV  do  not  occur.  In  the  case  where  a  gap 

is  present,  the  classical  method  will  find  the  desired  x'  which  maximizes  H(x), 
but  this  x'  does  not  maximize  F(x)  at  the  same  time.  F(x)  does  have  a 
stationary  point  at  x',  however.  Example  6  illustrates  a  problem  which  is 
solved  immediately  by  the  classical  method,  but  not  by  the  direct  method  unless 
modifications  are  introduced  to  eliminate  the  gap. 

(b)  The  line  H  =  Xg  +  F(x')  may  pass  through  the  payoff  set  without  being 
anywhere  tangent.  Such  solutions  correspond  to  the  relative  extrema  which 
are  not  absolute  extrema,  or  to  stationary  points  which  are  neither. 


Comment  4:  When  the  classical  method  obtains  the  absolute  extrema  the 
resulting  LMs  can  be  interpreted  as  partial  derivatives  of  maxH  or  minH 
in  analogy  to  the  direct  method.  This  is  suggested  by  the  following:  Multiply 
each  of  the  equations  F  =  0  by  d  and  sum,  obtaining  dF(x)  =  0,  and 

X  •  Xt 

1  1 

hence  dH(x)  =^Xjdg^(x).  Evaluating  this  latter  equation  at  the  extremizing 
point  x'  obtained  in  step  3,  we  have 

dH(x')  =£Ydg.(x'). 

Intuitively,  x'  is  a  function  of  C,  denoted  x'(C).  H(x')  is  therefore  H(x'(C)), 
which  is,  in  fact,  maxH(C)  or  minH(C)  since  x'  was  extremizing.  This, 
plus  the  equations  gj(x')  =  C.,  suggests  that  d(maxH(C))  =2j^jdCj  (or  rninH, 
as  desired)  and  hence  that  X  ^  is  the  partial  derivative  of  maxH(C)  (or  minH(C)) 
with  respect  to  Cy  as  desired. 
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VII.  HOMOGENEOUS  FUNCTIONS 

When  H(x)  and  g.(x)  are  homogeneous  functions,  a  good  deal  is  known 
about  the  resulting  functions  maxH  and  minH  .  In  particular,  the  single  con¬ 
straint  problem  becomes  quite  susceptible  to  attack  by  direct  method.  A  function 
f(x)  is  homogeneous,  of  degree  "p",  if  f(ax)  =  apf(x)  for  all  values  of  "a"  (for 
this  discussion,  "x"  will  be  treated  as  having  components  "x."  but  no  restrictions 
on  the^type  of  number  of  these  components  is  desired).  The  function  f(x,y)  = 
x+xy  is  homogeneous  oi  degree  3.  The  function  f(x,y,z)  =  max(x,y,z2/y)  is 
homogeneous  of  degree  1,  and  illustrates  a  discontinuous  homogeneous  function. 
The  homogeneity  property  imposes  a  certain  regularity  of  behavior  upon  the 
function,  nevertheless,  homogeneous  functions  may  he  exceedingly  complex  or 
otherwise  intractible.  Fortunately,  homogeneity  can  frequently  be  determined 
by  inspection  in  spite  of  the  complexity. 

An  "isobaric"  function  is  a  useful  generalization  of  a  homogeneous  function. 

A  function  f(x)  is  isobaric  if  a  constant  p  and  a  vector  q  exist  such  that 
f(a  x)  =  apf(x)  for  all  values  of  a ,  where  the  notation  ”aqx"  denotes  the  vector 

with  components  "a  Sc.".  The  function  f(x,y)  =  |  xy2  |  is  isobaric  since 

|  (a  x)  (ay)  |  -  a  |  xy  |,  i.e.,  p,qj,q2  =  4,2,1,  respectively.  It  is  easily 
seen  that  if  all  values  q.  are  equal,  the  isobaric  function  reduces  to  a  homogen¬ 
eous  function  of  degree  p/q.. 

Theorem  1:  Let  H(x)  be  homogeneous  of  degree  p ,  and  let  each  of  the 
functions  g.  (x)  be  homogeneous  of  degree  q.  .  If  the  function  maxH  (or  minH) 
exists,  it  is  isobaric,  and  maxH(aqC)=  apmaxH(C). 

Pro<^:  ^et  ^  subset  of  S  for  which  the  sector  equation  g(x)  =  C  holds. 

Let  S2  be  the  subset  of  S  for  which  g(x)  =  aqC.  Using  the  homogeneity  of 
each  function  g. ,  it  follows  that  if  x  is  a  point  in  Sj  then  ax  is  a  point  in 

S2  (multiply  both  sides  of  the  equation  g.(x)  =  C.  bya^).  Conversely,  any 

point  z  in  S2  can  be  written  as  ax  where  x  is  a  point  in  S1  .  It  follows 
therefore,  that 
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maxH(aqC)  =  max  H(z)  =  max  H(ax)  =  max  apH(x)  =  apmaxH(C). 
zeS2  xes]  xeSj 

Theorem  2:  If  f(x)  is  isobaric,  then  f(x)  has  a  derivative,  denoted  f’(x), 

along  the  curve  aqx,  where  a  is  a  parameter.  Furthermore,  f'(x)  =  (p/qx)f(x). 

Proof:  f'(x)  =  limit  f(aqx)  -  f(x)  =  limit  apf(x)  -  f(x)  =  f(x)  limit  ap  -  1  =  (p/qx)f(x). 
aq-»  1  aqx  —  x  aq->  1  aqx  -  x  x  aq-*laq-  1 

Corollary  1.  Under  the  conditions  of  theorem  1,  if  there  is  only  one  constraint 

then  maxH(C)  and  minH(C)  (if  they  exist)  are  of  the  form  ACP^q,  where  A 
is  a  constant. 

proof:  Using  theorems  1  and  2,  the  functions  maxH  and  minH  satisfy  the  dif¬ 
ferential  equation 

df(C)/dC  =  (p/ qC)f(C) 

which  has  the  general  solution  f(C)  =  ACP^q. 

Comment  1;  This  corollary  is  the  primary  result  of  this  section.  If  the  direct 
method  is  used,  only  one  guess  for  the  value  of  the  LM  need  be  made  since 
the  resulting  values  of  C  and  maxH(C)  can  be  used  to  evaluate  "A"  and  obtain 
the  entire  solution.  This  finds  application  in  the  classical  method,  also.  After 
the  partial  derivatives  have  been  equated  to  zero,  the  resulting  system  may  be 
impossible  to  solve  in  terms  of  the  LM  as  a  general  parameter.  It  suffices, 
however,  to  solve  the  system  for  only  one  value  of  the  LM  .  Example  7 
illustrates  these  ideas. 

Comment  2;  A  gap  or  constant  multiplier  can  be  immediately  spotted  by 
examining  p/q.  A  simple  technique  for  remedying  an  unfavorable  p/q  will  be 
presented  in  section  VIII. 

Comment  3;  The  proof  of  theorem  2  does  not  require  the  isobaric  property  for 
all  values  of  a ,  merely  for  values  of  a  in  some  neighborhood  of  a  =  1  . 

Similarly,  theorem  1  remains  valid  for  all  values  of  a  for  which  the  functions 
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involved  are  homogeneous.  Hence,  corollary  1  holds  providing  only  that  the 
functions  involved  are  all  homogeneous  for  values  of  a  within  some  neighborhood 
of  a  =  1  .  The  utility  of  this  is  that  many  functions  occurring  in  practical  problems 
are  homogeneous  for  restricted  values  of  a,  e.g.,  functions  involving  absolute 
values  may  require  a  to  be  non -negative. 


Comment  4;  Functions  which  are  piecewise  defined  as,  for  example, 


f(x,y)=  { 


x  +  3y,  x  s  2 
2x  +  y,  x  <  2 


} 


are  not  necessarily  homogeneous  along  the  boundaries  common  to  the  various 
pieces.  This  is  in  spite  of  the  fact  that  the  degree  of  homogeneity  may  be  the 
same  within  each  separate  piece.  The  problem  is  that  of  proper  continuity  at  the 
boundary  (see  comment  3,  above).  Homogeneous  functions  may  be  discontinuous, 
but  their  discontinuities  must  be  of  a  rather  specific  type;  in  particular,  the 
homogeneous  function  must  be  continuous  along  all  radial  curves . 

There  is  an  analogous  result  for  the  multi -constraint  case. 

Lemma  1;  If  the  function  maxH  is  differentiable,  it  obeys  the  first  order  linear 
partial  differential  equation 

I  Vifx.  =  pf 

i 

proof:  Differentiate  both  sides  of  the  equation  f(a\)  =  aPf(x)  with  respect  to 
a ,  and  let  a  go  to  1 . 


Comment  1:  The  differentiability  of  the  function  f  must  be  assumed.  Theorem 
2  guarantees  the  existence  of  only  one  directional  derivative;  the  part  'als  may  not 
exist . 


Comment  2:  This  partial  differential  equation  is  readily  solved,  giving 
G(fq/xP)  =  0 

where  G  is  an  arbitrary  function.  If  a  suitably  well  behaved  boundary  condition 
were  abailable  in  the  form  maxH(C)  =  M(t)  along  a  known  curve  C(t),  where 
t  is  a  parameter,  then  the  arbitrary  function  G  could  be  analytically  determined. 
In  most  practical  applications,  the  boundary  conditions  will  be  analytically 
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unsuitable,  or  unknown.  In  the  former  case,  numerical  integration  of  the 
partial  differential  equation  may  be  successful;  in  the  latter  case,  the  direct 
method  may  be  tried.  It  may  be  possible  to  approximate  a  boundary  condition 
arbitrarily  closely  by  varying  the  LMs  so  as  to  sketch  out  M(t)  along  some 
arbitrary  curve  C(t). 
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VIII.  GENERALIZATIONS 


In  applications  of  the  classical  method,  it  is  common  to  maximize  log(H(x)) 

2 

or  (H(x))  if  these  are  more  tractable  analytically,  since  "maximizing  these 
is  the  same  as  maximizing  H(x)" .  In  general  it  is  true  that  if  G  is  any 
monotonically  increasing  (strictly)  function,  then  G(H(x))  attains  its  maximum 
at  the  same  point  x'  as  the  function  H(x).  This  fact  is  of  fundamental  impor¬ 
tance  in  the  direct  method  because  the  function  G  changes  the  shape  of  the 
payoff  set.  A  proper  choice  of  G  can  eliminate  a  gap  or  can  eliminate  the 
"constant  multiplier"  without  changing  x*  (example  6). 

The  constraint  functions  may  also  be  modified  into  G(g(x))  =  C.  This 
modification  also  changes  the  shape  of  the  payoff  set  and  can  be  used  instead  of., 
or  in  addition  to,  modifying  H(x).  These  two  types  of  modifications  have 
inverse  effects  on  the  shape  of  the  payoff  set,  however  (example  6). 

The  single  constraint  problem  with  homogeneous  functions  can  often  be 
effectively  handled  using  these  concepts.  If  the  ratio  of  degrees  of  homogeneity 
(p/q  of  the  preceeding  section)  is  unfavorable,  G  can  be  chosen  homogeneous  of 
the  proper  degree  to  guarantee  that  no  gap  exists  (example  6). 

It  is  sometimes  possible  to  retain  the  LMs  as  parameters  in  the  direct 
method  and  still  perform  the  maximization  required  in  step  2,  section  II.  This 
requires  that  x'  be  obtainable  as  a  function  of  the  parameter  A..  If  non- 
differentiable  functions  are  involved,  then  x’(X)  will  probably  be  non -analytic 
and  must  be  examined  point  at  a  time.  Example  8  illustrates  a  case  where 
the  classical  method  will  not  work,  but  the  direct  method  applies;  the  maximi¬ 
zation  (minimization  in  this  case)  can  be  done  analytically,  retaining  the  LM 
throughout  as  a  parameter. 
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APPENDIX  A 


MISCELLANEOUS  EXAMPLES 

Example  1;  This  problem  illustrates  the  direct  method  when  the  set  S  is  a 
collection  of  discrete  points. 

Maximize  |x  —  2y  I  subject  to  x  +  y  =3  where  S  consists  of  the 
collection  of  points  (0,1),  (2,1),  (4,1),  (3,0),  (1,2). 

Step  1:  F(x,  y)  =  lx2  -  2y  |  -  X  (x  +  y2)  . 

§_teP  2:  is  apparent  that  in  order  to  maximize  F(x,  y)  we  must  plug  in  each  of 

the  points  in  S  in  turn  and  evaluate  F(x,  y)  at  each  point.  Further,  we  can¬ 
not  investigate  F(x,  y)  numerically  for  a  maximum  so  long  as  the  LM  is  a 
parameter.  It  is  necessary  to  insert  a  numerical  value  for  X,  then  proceed 


with  that  value  into  step  3. 

Listed  below  are  the  results  of  three 

guesses  for 

X 

Maximizing 
point  (x') 

H(x') 

g(x’) 

F(x') 

10 

(0,  1) 

2 

1 

-8 

1 

(4,  1) 

15 

5 

9 

3 

(3,0) 

9 

3 

0 

The  first 

guess,  X  =  10,  gave  a  maximum  of 

—8  to 

F(x,  y),  the 

maximizing 

point  being  (0, 1).  Proceeding  into  step  3,  g(0,  1)  turns  out  to  be  1.  Since  the 
original  problem  specifies  the  constraint  g(x,  y)  =  3,  the  value  of  10  for  the  LM 
is  unsatisfactory  and  another  guess  must  be  made.  In  section  II it  will  be  made 
clear  that  the  LM  must  be  decreased  in  order  to  increase  the  resulting- value 
°f  g(x,  y).  A  guess  of  1  for  X  produces  a  constraint  value  of  5,  i.e.,  the  LM 
has  been  decreased  too  much.  A  guess  of  3  for  X  finally  produces  g(x,  y)  =  3, 

as  desired.  According  to  the  main  theorem,  section  I,  the  point  (3, 0)  maximizes 
H(x,  y)  subject  to  g(x,  y)  =  3. 

Similar  problems  involving  larger  sets  S  ,  more  constraints,  and  more 
complex  functions  can  be  solved  in  exactly  this  way,  using  a  computer  to 
evaluate  the  functions . 
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Example  2:  This  problem  illustrates  how  the  function  "minH"  comes  into  being. 

It  is  desired  to  find  the  minimum  distance  from  the  origin  of  the  x,  y  plane 

to  the  curve  defined  by  xy  =  b,  for  b  >  0  .  The  problem  is  trivial  and  can  be 

solved  readily  in  a  variety  of  ways;  by  sketching  the  graph  of  xy  =  b  for  some 

1/2 

value  of  b  it  will  become  obvious  that  the  solution  must  be  x  =  y  =  (b)  . 

2  2  1/2 

For  this  problem  H(x,  y)  =  (x  +  y  )  .  Plugging  in  the  solution,  H(x,  y)  be¬ 
l/2  1/2 

comes  (b  +  b)  '  or  (2b)  '  ,  which  is  no  longer  a  function  of  x  and  y,  but 
of  the  parameter  b  .  This  resulting  function  of  b  is  minH(b) . 


♦ 
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Example  3:  This  problem  illustrates  how  the  geometric  interpretations  of  the 
functions  minH(C)  and  g(x)  may  be  used  to  estimate  the  values  of  the  LMs. 

In  this  example,  these  considerations  permit  an  exact  solution. 

It  is  desired  to  find  the  minimum  distance  between  the  line  x  +  y  =  a  and 
the  square  max(|x|,  |yj)  =  b,  where  the  values  of  a  and  b  are  such  that  the 

9 

two  figures  are  not  intersecting.  In  this  problem,  H(x. ,  y  ,  x0,  y~)  =  ((y  —  y0) 

2  1/2  1  1  L  L  1  L 

(X1  “  x2)  )  '  .  g1(x1.y1.x2.y2)  =  *1  +  y{>  and  g2(x1,y1,x2,y2)  =  max(|x2|,  |y2  | 

where  the  subscript  "1"  refers  to  the  line,  and  "2"  refers  to  the  square.  The 
reader  is  requested  to  examine  the  representative  figure  below  and  use  the 
interpretation  of  the  LMs  as  partial  derivatives,  along  with  the  geometric  in¬ 
formation,  to  evaluate  the  LMs  directly. 


Solution:  The  function  minH(a,  b)  has  the  geometric  interpretation  of  minimum 
distance,  hence  is  the  derivative  of  this  distance  with  respect  to  "a",  and 
\2  is  the  derivative  with  respect  to  "b" .  An  increase  of  a  by  one  unit  causes 

1/2  1/2 

an  increase  in  minH(a,b)  by  1/(2)  '  units,  hence  =  1/(2)  .  Similarly, 

an  increase  of  b  by  one  unit  causes  a  decrease  in  minH(a,  b)  by  one  unit, 
hence  =  —  1  . 

Note  that  the  values  of  the  partial  derivatives  are  independent  of  both  a 
and  b.  This  is  untypical;  usually  the  LMs  are  functions  of  all  of  the  constants. 


Example  4:  This  is  a  cell  problem  in  which  H(x)  is  non-differentiable. 

Maximize  H(x)  =  H^x^)  +  H2(x2)  subject  to  the  constraint  g(x^,  x2)  = 
xx  +  x2  =  5,  where  H^x^  =  x^lO  -  x^  and  H2(x2)  is  as  shown. 


and  each  of  the  terms  in  brackets  can  be  maximized  separately.  The  first 
bracket  can  be  maximized  analytically,  yielding  a  maximum  at  Xj  =  5  —  X/2. 
The  second  bracket  is  not  differentiable  at  all  points  and  must  be  maximized 
by  considering  separate  values  of  \  .  This  can  be  done  by  inspection,  in  this 
example,  since  the  graph  of  H2(x2)  is  available;  however,  if  H2  were  more 
complex,  it  might  be  necessary  to  divide  up  the  x2  axis  into  small  increments 
and  let  a  computer  search  for  the  maximum.  The  table  below  summarizes  the 


results  of  various  Jesses 

of  the  LM . 

\  xi 

x2 

xl+x2 

H1(x1,x2) 

*VX1’  x2^ 

H(xrx2) 

F(xrx2) 

2  4 

2-5 

6-9 

24 

10-16 

34-40 

22 

3  3.5 

2 

5.5 

22.75 

10 

32.75 

16.25 

4  3 

2 

5 

21 

10 

31 

11 
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The  point  x’  =  (3, 2)  satisfies  the  constraint  g^,  x2)  =  x:  +  x2  =  5,  hence 
it  is  the  desired  point,  giving  a  maximum  value  of  31  to  H(x1>  x2).  Note  that 
for  \  -  2,  the  maximizing  point  x*  is  not  unique  since  x2  may  vary  between 
2  and  5,  producing  a  corresponding  variation  in  Hfx^  x2)  (but  not  in  Ffx^  x2)). 
This  is  the  "constant  multiplier"  discussed  in  section  IV.  Note  that  as  the  LM 
is  varied  from  3  to  4,  x2  remains  constant  at  2.  This  is  because  H2  is  non- 
differentiable,  and  is  discussed  in  section  IV. 

As  the  LM  is  varied  from  2  to  4,  various  values  of  g(x1>x2)  and  their 
accompanying  values  of  H(x1>x2)  are  also  generated.  The  plot  of  Hfx^x^vs 

g<xr  x2^  *s  discussed  in  detail  in  sections  III  and  IV,  and  is  fundamental  to  the 
study  of  LMs. 

This  problem  is  handled  in  a  straightforward  way  using  the  direct  method. 
The  classical  method  cannot  handle  it  since  H2(x2)  is  not  differentiable  at  the 
maximizing  point. 


V 


Example  5:  This  problem  illustrates  a  straightforward  application  of  the 
classical  method. 

Find  the  minimum  distance  from  the  origin  to  the  subspace  determined  by 
the  intersection  of  the  two  hyperplanes 

a. x  =  C1 

b. x  =  C2 

where  the  dot  product  notation  has  the  usual  interpretation 
a.x  =  ^  a.x.  . 

t 

Solution:  For  this  problem,  H(x)  =  (x.x)1//2,  however  the  square  of  this  will  be 
minimized  instead  since  it  is  more  tractable  analytically.  This  technique  is 
discussed  in  section  VIII. 

Step  1;  F(x)  =  x.x  -  Xja.x  -  ^b.x 

Step  2:  Equating  the  partial  derivatives  to  zero  gives  the  vector  equation 
2x  —  Xj  a  —  X^b  *  0 

from  which  x  can  immediately  be  expressed  as  a  function  of  the  LMs  . 
x=  l/2(Xia  +  X2b) 

Step  3:  This  vector  x  can  be  plugged  into  the  constraint  equations,  giving: 
\1a.a+X2a.b  =  2C1 

Xjb.a  +  X2b.b  =  2C2 

Since  the  dot  products  can  be  calculated  directly,  the  above  equation  in  the  two 
unknowns  X1 ,  X2  can  be  solved  by  Cramer's  rule  to  give  the  LMs  as  functions 
of  C^,  C2  .  The  determinant  D  is 

(a. a)  (b.b)  -  (a.b)2 
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which,  by  the  Cauchy -Schwarz  inequality,  is  non-zero  unless  the  vectors  a 
and  b  are  colinear.  In  this  case,  the  two  planes  are  parallel  and  intuition 
agrees  that  there  is  no  solution.  Solving  for  the  LMs 
\  =  2(0^  -  C2  a . b)/D 

\2  =  2(  — C  j  a .  b  +  C2  a .  a)/D 

These  values  can  be  plugged  into  the  expression  for  x ,  giving  x  as  a  function 
of  Cj,  C2  .  The  expression  x.x  can  then  be  evaluated  giving,  finally, 
minHfCj,  C2). 

A  faster  method,  however,  is  to  take  the  equation  for  x  ,  dot  both  sides 
with  "x",  and  evaluate  at  the  minimizing  point,  giving: 
minH(Cr  C2  *  l/2(X1C1  +  X^) 

C12b-b-2C1C2a.b  +  C22  a.a 

D - 

This  savings  in  labor  is  always  possible  if  the  functions  H(x)  and  g.(x)  are 
homogeneous  (section  VIII) .  1 
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Example  6:  This  problem  can  be  solved  immediately  by  the  classical  method, 
but  cannot  be  solved  by  the  direct  method  unless  one  or  more  of  the  functions 
is  modified  to  eliminate  the  "gap".  A  few  possible  modifications  are  illustrated. 

2  2  1/2 

Minimize  (x  +  y  )  '  subject  to  the  constraint  g(x,  y)  =  xy  =  b,  with 
b  >  0.  S  is  the  first  quadrant  of  the  x,  y  plane.  The  classical  method  gives 
the  solution  x  =  y  =  (b)1/2,  \  =  l/(2b)1/2,  minH(b)  =  (2b)1/2.  The  payoff  set 
in  the  g,  H  plane  is  shown  below. 


The  function  minH(b)  is  concave  downward,  hence  the  entire  positive  g 
axis  (except  the  origin)  is  a  "gap”  and  the  direct  method  will  fail  (the  inequality 
of  section  III  is  reversed  here  since  a  minimum  is  desired). 

We  can  modify  the  shape  of  the  payoff  set  so  as  to  eliminate  the  gap  by 
modifying  either  H(x,y),  g(x,  y)  or  both,  appropriately.  If  G(H)  is  a 
monotonically  increasing  function  of  H  in  the  region  of  interest,  then 
G(H(x,  y))  can  be  used  in  place  of  H(x,  y>.  Furthermore,  the  gap  will  be 
reduced  or  eliminated.  Alternately,  if  G*(g)  is  a  monotonically  decreasing 
function  of  g,  then  G*(g(x,  y))  can  be  used  in  place  of  g(x,  y)  with  similar 
results.  The  table  on  the  following  page  summarizes  the  effect  of  various 
functions  G  and  G* . 
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G(H(x,  y)) 

G*(g(x,y))  =  b 

X 

minG(H(b)) 

X 

2  2 
x  +  y 

xy  =  b 

<b)1/2 

2b 

2 

(x  +  y  )  ' 

X) 

ii 

es 

S'. 

b 

(2)1/2b 

(2)l/2 

2  ,  2 
x  +  y 

(xy)1/2  =  b 

b 

2b2 

4b 

,  2  ,  2,2 
(x  +y  ) 

xy  =  b 

<b)1/2 

4b2 

8b 

The  first  row  shows  the  effect  of  using  G(H)  =  H2,  while  retaining  the 
same  constraint  function.  The  minimizing  point  is  still  at  x  =  y  =  (b)1/2, 
producing  minG(H(b))  =  2b  or  minH(b)  =  (2b)1/2  as  expected.  The  LM, 
however,  has  changed  from  l/(2b)1//2  to  2,  i.e.,  X  is  constant  for  all 
values  of  b;  we  now  have  the  "constant  multiplier".  Since  the  LM  is  constant 
for  all  values  of  b,  the  graph  of  minG(H(b))  is  a  straight  line  with  slope  2, 
i.e. ,  the  graph  is  no  longer  concave  downward,  but  is  a  straight  line.  This 
is  progress  in  the  right  direction.  If  instead,  we  use  G(H)  =  H,  but 
G*(g)  =  g  as  in  the  second  row,  the  result  is  similar,  a  constant  multiplier, 
but  with  a  different  value.  The  last  two  rows  show  the  effects  of  combining  G 
and  G*.  The  resulting  functions  minG(H(b))  are  concave  upward,  no  gap 
exists,  and  the  direct  method  would  solve  the  problem  easily. 

Since  this  is  a  single  constraint  problem  involving  homogeneous  functions, 
the  results  of  using  the  functions  G  and  G*  above  can  be  read  off  directly  by 
inspection,  without  having  to  solve  the  problem  first.  This  would  permit  the 
analyst  to  make  an  appropriate  choice  of  G  and  G*  prior  to  attempting  the 
direct  method.  Using  corollary  1,  section  VII,  "p/q"  for  each  of  the  examples 
in  the  above  table  is  1, 1,  2,  2  (descending  order).  These  match  with  the 
exponents  of  "b"  in  the  column  headed  "minG(H(b))M . 


Example  7:  This  problem  illustrates  some  of  the  simplifications  which  may  be 
possible  in  the  direct  method  if  the  functions  involved  are  homogeneous. 

J}  n 

It  is  desired  to  minimize  H(x)=)  a.x.  subject  to  g(x)  =  ^  |x.  |1//2  =  C. 

S  is  all  of  n -space.  1  1 

O 

Solution:  The  n -space  surface  g(x)  =  C  has  cusps  at  +C  on  each  of  the 
coordinate  axes.  A  little  reflection  about  the  shape  of  this  surface  will  make 
it  clear  that  the  minimizing  point(s)  must  be  one  (or  more)  of  these  cusps,  at 
which  g(x)  is  not  differentiable.  Hence,  the  classical  method  cannot  be  ex¬ 
pected  to  work.  If  n  is  small,  it  may  be  practical  to  locate  the  desired  cusp 
by  hand  calculations,  but  if  n  is  large,  and  several  values  of  C  must  be 
examined,  computer  assistance  will  be  welcome. 

The  functions  H(x),  g(x)  are  homogeneous  of  degree  1,  1/2,  respectively, 

hence  p/q  =  2,  and  the  direct  method  will  work.  The  function  minH(C)  is  of 
2 

form  AC  ,  so  that  any  guess  greater  than  zero  for  the  LM  should  produce  a 
meaningful  result. 

After  making  a  guess  for  the  LM  (X  =  1,  say),  a  computer  may  easily  be 
programmed  to  evaluate  F(x)  at  each  of  the  cusps,  determine  the  minimizing 
cusp,  and  print  out  the  resulting  C’  and  minH(C’).  These  values  are  plugged 
into  the  equation 

minH(C)  =  AC2 

giving  the  result  that  A  =  minH(C')/C’2.  This  solves  the  problem  for  all 
values  of  C. 
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Example  8:  This  problem  illustrates  the  direct  method  in  which  the  LM  re¬ 
mains  as  a  parameter  throughout  the  minimization. 

2  2 

Find  the  minimum  of  H(x,  y)  =  x  +  y  subject  to  the  constraint  g(x,  y)  = 
y  -  (x-l)3/2  =  2. 

Step  1;  F(x,  y)  =  x2  +  y2  -  \y  +  \(x-l)3/2 

SteP  2:  F(x»y)  can  be  minimized  with  respect  to  x  by  inspection,  obtaining 
x  =  1  as  the  minimizing  value  (note  that  x  cannot  be  less  than  1  because  of 
the  exponent).  F(x,y)  reduces  therefore  to  y2  -  \y  +  1  which  is  parabolic 
in  y.  This  function  has  a  minimum  at  y  =  X/2. 

SteP  3:  Hence  g(x,y)  becomes  \/2.  By  the  main  theorem,  section  I,  we  know 
that  the  point  x’  =  (1,  X/2)  provides  an  absolute  minimum  for  H(x,  y).  By 
setting  \  =  4,  the  required  constraint  g(x,y)  =  2  is  satisfied. 

Although  it  is  not  obvious  by  inspection  of  the  problem,  the  classical  method 
will  fail  to  solve  this  problem  since  the  partial  derivative  of  F(x,y),  with 
respect  to  x,  does  not  exist  at  the  minimizing  point.  The  constraint  function 
g(x,  y)  is  in  fact  discontinuous  at  the  minimizing  point. 

There  is  no  need  to  guess  a  value  for  the  LM  in  step  2  since  the  minimiza¬ 
tion  done  there  holds  for  all  positive  values  of  the  LM  (at  least)  rather  than  a 
single  value.  Normally,  when  discontinuous  functions  are  involved,  the  minimiza¬ 
tion  must  be  done  numerically,  which  requires  that  the  LM  be  assigned  some 
definite  value . 


A-ll 

(REVERSE  BLANK) 


_ Unclassified  _ 

_ 

DOCUMENT  CONTROL  DATA  •  R  &  D 

<■/„ -s  -III, ■■■iIi.iii  ill  title,  limty  abstract  a, el  itulexm,)  .mnutatiun  nm.'l  hr  entered  when  Hi,-  overall  report  is  classilied) 

C^emtio^s'Ev^onGr^p,  Center  for  Naval  Analyses  U^asslS*  ~ " “ 
of  the  Franklin  Institute  _ unciassitied _ 

2b.  GUOUP 


’  REPORT  TITLE  “  - - 

The  Lagrange  Multiplier  —  A  Heuristic  Presentation 


None 


4  °r>5C  RIP  Tl  V  E  NO  TE  s  (TyPe  of  report  and  inclusive  dales) 

Research  Contribution 

5  Au  THORiS)  (First  name,  middle  initial ,  last  name) 

Nunn,  W.R. 


6  REPOR  T  D A  TE 

31  October  1966 

0a.  CONTRACT  OR  GRANT  NO 

b.  PROJ  EC  T  NO. 


NONR  3732  (00) 


7a.  TOTAL  NO.  OF  PAGES  7b.  NO .  OF  REFS 

35  2 

9fl.  ORIGINATOR'S  REPORT  NUMBER(S)  "  ~  — 

Operations  Evaluation  Group  Research 
Contribution  No.  76 

9fc  JhJsH«p0rfJ  P0RT  NO,SI  (Ar,y  °,,,er  numbers  that  may  be  assigned 

None 


10  DISTRIBUTION  STATEMENT 

Distribution  of  this  document  is  unlimited. 

• 

ii.  supplementary  notes 

None 

Office  of  Naval  Research 

Department  of  the  Navy 

Washington,  D.C.  20350 

13.  ABSTRACT  - 

Many  problems  in  operations  research  require  the  maximization  or  minimization  of  a 
suitable  payoff  function  subject  to  various  constraints.  Lagrange  multipliers  are  classically 
used  for  this  type  of  problem,  however,  the  treatment  given  this  technique  by  most  texts 
requires  that  the  payoff  and  constraint  functions  be  at  least  differentiable  at  the  extremizing 
point.  This  paper  shows  that  the  Lagrange  multiplier  concept  can  be  independent  of  dif¬ 
ferentiability  or  even  continuity  of  the  functions  involved.  It  also  gives  the  reader  a 
geometric  insight  into  the  workings  of  the  multiplier.  Simplifications  possible,  if  the 
functions  involved  are  homogeneous,  are  displayed.  The  treatment  is  heuristic  rather 
than  formally  rigorous . 

y  v 

(Unclassified) 


DD  ,”"“..1473  (PAGE  1 ) 

S/N  0)01.807-6801 


Unclassified 
Security  Classification 


Unclassified 

Security  Classification 


LINK  A 


Lagrange  multiplier 

maximization-minimization 

constraints 

payoff  set 

differentiability 

homogeneous  functions 

continuity 


(Unclassified) 


DD 

(PAGE-  2) 


,1473  ( BACK) 


Unclassified 

Security  Classification 


